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EXECUTIVE  SUMMARY 


The  Environmental  Symptoms  Questionnaire  (ESQ-lll)  is  a  validated  67-item 
paper  and  pencil  questionnaire  utilized  to  assess  symptoms  of  acute  mountain  sickness 
(AMS)  in  individuals  exposed  to  altitude.  Problems  encountered  with  administering  the 
ESQ-lll  include  1)  lengthy  time  to  complete,  2)  inaccurate  answers  due  to  boredom,  3) 
intentional  or  unintentional  skipping  of  questions  and  4)  multiple  answers  to  the  same 
question  due  to  stray  marks.  In  order  to  eliminate  these  problems,  a  shortened  (11- 
item),  electronic  version  of  the  ESQ  was  developed  that  could  be  administered  on  a 
hand-held  computer  (IPAQ  5550,  Hewlett  Packard,  Palo  Alto,  CA).  The  purpose  of  this 
study  was  to  validate  the  shortened  electronic  version  of  the  ESQ  to  assess  AMS. 
Thirty-four  volunteers  (28±1  yrs;  74±2  kg;  174±3  cm;  means  ±SE)  were  given  both  the 
paper  and  pencil  and  the  shortened  electronic  version  of  the  ESQ  to  complete  one  after 
the  other  at  residence  altitude  (RA),  and  after  24-h  (PP24),  48-h  (PP48),  and  72-h 
(PP72)  exposure  to  4300  m  on  the  summit  of  Pikes  Peak  (PP).  The  AMS-Cerebral 
(AMS-C)  factor  score  was  calculated  from  responses  to  the  same  1 1  items  for  each 
version  of  the  ESQ.  If  the  AMS-C  score  was  >0.7  then  the  individual  was  classified  as 
having  AMS.  There  were  no  differences  in  the  severity  of  AMS  between  the  paper  and 
pencil  and  electronic  versions  of  the  ESQ  at  RA  (0.05±0.01  vs.  0.03±0.01),  PP24 
(0.85±0.16  vs.  0.80±0.16),  PP48  (0.61  ±0.1 5  vs.  0.58±0.15),  and  PP72  (0.34±0.09  vs. 
0.35±0.09).  There  were  also  no  differences  in  the  incidence  of  AMS  between  the  paper 
and  pencil  and  electronic  versions  of  the  ESQ  at  RA  (0%  vs.  0%),  PP24  (35%  vs.  38%), 
PP48  (26%  vs.  26%),  and  PP72  (18%  vs.  18%).  The  relationships  between  the  severity 
of  AMS  calculated  from  the  two  versions  of  the  ESQ  at  RA  (r=0.43;p=0.01 ),  PP24 
(r=0.92;p=0.0001),  PP48  (r=0.82;p=0.0005),  and  PP72  (r=0.95;p=0.0001)  were  highly 
significant.  The  relationships  between  the  incidence  of  AMS  calculated  from  the  two 
versions  of  the  ESQ  at  RA  (k=0.90;p=0.01),  PP24  (k=0.90;p=0.01),  PP48  (k=0.91; 
p=0.01),  and  PP72  (k=0.92;  p=0.005)  were  also  highly  significant.  Qur  findings  suggest 
that  the  shortened  (1 1 -item)  electronic  version  can  be  substituted  for  the  67-item  paper 
and  pencil  version  of  the  ESQ  in  order  to  assess  the  severity  and  incidence  of  AMS  at 
4300  m. 
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INTRODUCTION 


Acute  Mountain  Sickness  (AMS)  is  a  syndrome  that  is  characterized  by 
headache,  insomnia,  anorexia,  nausea,  dizziness,  and  fatigue,  but  without  abnormal 
neurological  findings  (11,  22).  The  severity  and  incidence  of  AMS  is  primarily  related  to 
the  initial  altitude,  the  rate  of  ascent,  the  altitude  reached,  and  the  duration  of  exposure 
to  altitude  (9,  13,  16,  21,  30).  Additional  factors  that  affect  the  severity  and  incidence  of 
AMS  are  the  degree  of  hypoxemia  (2,  5,  10,  18),  level  of  physical  exertion  performed  (8, 
24),  individual  susceptibility  (25,  31),  and  degree  of  prior  altitude  acclimatization  (12, 

15).  Symptoms  of  AMS  typically  become  evident  in  the  first  few  hours  of  altitude 
exposure  and  reach  peak  severity  in  24  to  48  h  (22).  The  chief  significance  of  AMS  is 
that  people  rapidly  exposed  to  altitude  may  be  completely  incapacitated  in  the  first  few 
days  at  altitude  (22).  Additionally,  in  a  few  individuals,  AMS  may  progress  to  life- 
threatening  high-altitude  cerebral  edema  or  high-altitude  pulmonary  edema,  where 
evacuation  is  required. 

The  ability  to  measure  AMS  accurately,  therefore,  is  critically  important  in 
individuals  ascending  to  high  altitudes  for  work  or  recreation.  AMS  is  traditionally 
measured  using  a  validated  67-item  paper  and  pencil  Environmental  Symptoms 
Questionnaire  (ESQ-III)  (28).  The  AMS-Cerebral  (AMS-C)  factor  score  is  calculated 
from  answers  to  1 1  of  the  67  questions  to  assess  the  severity  of  AMS.  If  the  AMS-C 
score  is  >0.7  then  the  individual  is  classified  as  having  AMS  (e.g.,  sick  versus  not  sick). 
Despite  its  usefulness  in  assessing  AMS,  problems  encountered  with  administering  the 
ESQ-III  include:  1)  lengthy  time  to  complete,  2)  inaccurate  answers  due  to  boredom,  3) 
intentional  or  unintentional  skipping  of  questions  and  4)  multiple  answers  to  the  same 
question  due  to  stray  marks.  Although  a  shorter  five-item  paper  and  pencil 
questionnaire  (Lake  Louise  (LL)  AMS  Scoring  System)  exists  (23)  and  has  been 
validated  against  the  ESQ-III  (17,  29),  the  LL  questionnaire  tends  to  overestimate  AMS 
compared  to  the  ESQ-III  (1 , 27).  In  order  to  eliminate  administrative  problems  with  the 
paper  and  pencil  version  of  the  ESQ-III  and  overestimation  of  AMS  by  the  LL 
questionnaire,  a  shortened  (11 -item),  electronic  version  of  the  ESQ  was  developed  that 
could  be  administered  on  a  hand-held  computer.  The  purpose  of  this  study  was  to 
validate  the  shortened  electronic  version  of  the  ESQ  to  assess  AMS. 

METHODS 


SUBJECTS 

Thirty-four  sea-level  (n=17)  and  moderate-altitude  (n=17)  residents  (27  men,  7 
women)  with  a  mean  (±SE)  age,  body  weight,  and  height  of  28±1  yr,  74±2  kg,  174±3 
cm,  respectively,  completed  this  study.  Sea-level  and  moderate-altitude  residents  were 
recruited  in  order  to  measure  a  wide  range  of  AMS  during  exposure  to  4300  m.  The 
sea-level  residents  were  recruited  from  the  Palo  Alto,  CA  area  (15  m)  while  the 
moderate-altitude  residents  were  recruited  from  the  Colorado  Springs,  CQ  area  (-2000 
m).  All  volunteers  received  medical  examinations,  and  none  had  any  condition 
warranting  exclusion  from  the  study.  All  tested  within  normal  ranges  for  pulmonary 
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function.  All  had  normal  hemoglobin  concentration  and  serum  ferritin  levels.  Each 
gave  written  and  verbal  acknowledgment  of  their  informed  consent  and  was  made 
aware  of  their  right  to  withdraw  without  prejudice  at  any  time.  Investigators  adhered  to 
the  policies  for  protection  of  human  subjects  as  prescribed  in  Army  Regulation  70-25, 
and  the  research  was  conducted  in  adherence  with  the  provisions  of  45  CFR  Part  46. 

PROTOCOL 

Design 

This  study  utilized  a  two  (version)  x  four  (condition)  repeated-measures  counter¬ 
balanced  experimental  design.  The  two  ESQ  test  versions  were  defined  as  paper  and 
pencil  and  electronic.  The  four  test  conditions  were  defined  as  residence  altitude  (RA), 
and  following  24-h  (PP24),  48-h  (PP48),  and  72-h  (PP72)  exposure  to  4300  m  on  the 
summit  of  Pikes  Peak  (PP).  All  volunteers  were  rapidly  (^  h)  exposed  to  4300  m  after 
measurements  at  their  RA.  Each  test  volunteer  completed  each  of  the  two  versions  of 
the  ESQ  one  right  after  the  other  in  a  counterbalanced  fashion  (e.g.,  half  completed  the 
paper  and  pencil  version  first,  while  half  completed  the  electronic  version  first)  for  each 
of  the  four  test  conditions. 

Altitude-Illness  Assessment 

Severity  and  incidence  of  AMS  symptoms  were  determined  from  information 
gathered  using  both  the  67-item  paper  and  pencil  and  the  shortened  (11 -item) 
electronic  ESQ.  The  paper  and  pencil  ESQ  is  a  self-reported,  67-item  inventory 
designed  to  quantify  symptoms  induced  by  altitude  and  other  stressful  environments 
(28).  Symptom  severity  is  self-rated  on  a  scale  of  0  to  5,  with  a  score  of  “0”  indicating 
the  absence  of  symptoms  and  “5”  indicating  maximum  intensity  of  the  symptoms.  A 
weighted  average  of  cerebral  symptoms  (headache,  lightheaded,  dizzy)  designated 
“AMS-C”  was  calculated  from  1 1  of  the  67  items  to  assess  the  severity  of  AMS.  AMS 
was  judged  to  be  present  if  an  individual’s  AMS-C  score  was  >0.7.  The  effectiveness 
of  AMS-C  scores  in  identifying  individuals  with  AMS  has  been  previously  reported  and 
validated  (28).  The  shortened  electronic  ESQ  consisted  of  the  same  1 1  items  from  the 
paper  and  pencil  ESQ  that  are  used  to  determine  AMS-C,  and  was  administered  on  a 
hand-held  computer  (IPAQ  5550,  Hewlett  Packard,  Palo  Alto,  CA).  The  incidence  of 
AMS  (%)  for  both  versions  of  the  ESQ  was  defined  as  the  number  of  test  volunteers 
who  achieved  or  exceeded  the  AMS-C  criterion  score  value  at  each  time  point  divided 
by  the  total  number  of  test  volunteers  measured  at  that  time  point. 

STATISTICS 

A  two  (version)  x  four  (condition)  repeated-measures  ANQVA  and  Cochran’s  Q 
was  used  to  analyze  differences  in  the  severity  and  incidence  of  AMS,  respectively, 
between  test  versions  and  test  conditions.  Pearson-product  correlation  coefficients  and 
kappa  statistics  were  calculated  to  determine  the  relationships  between  the  severity  and 
incidence  of  AMS-C  scores,  respectively,  between  the  two  versions  of  the  ESQ  in  each 
of  the  four  test  conditions.  The  AMS-C  difference  scores  between  the  two  test  versions 
(paper  and  pencil  -  electronic)  were  calculated  in  each  of  the  four  test  conditions  and 
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compared  to  zero  as  well  as  to  each  other.  Individual  difference  scores  of  zero  would 
indicate  that  there  were  no  differences  between  the  two  ESQ  test  versions,  while 
negative  difference  scores  represent  overestimations  of  AMS-C  by  the  electronic 
version  and  positive  difference  scores  indicate  underestimations  of  AMS-C  by  the 
electronic  version.  The  limits  of  agreement  were  calculated  for  each  of  the  difference 
scores  in  each  of  the  test  conditions  and  Bland-Altman  plots  were  constructed  (3). 
Individual  difference  scores  that  have  a  tight  prediction  interval  around  zero  indicate 
greater  agreement  between  the  two  versions  of  the  ESQ.  Significant  main  effects  and 
interactions  were  analyzed  using  Tukey’s  least  significant  difference  test.  Statistical 
significance  was  set  at  P  <  0.05.  All  data  are  presented  as  means  ±  SE. 

RESULTS 


ALTITUDE-ILLNESS  ASSESSMENT 

The  AMS-C  scores  measured  using  the  paper  and  pencil  and  electronic  versions 
of  the  ESQ  at  RA  (0.05±0.01  vs.  0.03±0.01),  PP24  (0.85±0.16  vs.  0.80±0.16),  PP48 
(0.61  ±0.1 5  vs.  0.58±0.15),  and  PP72  (0.34±0.09  vs.  0.35±0.09)  were  not  significantly 
different  in  any  of  the  four  test  conditions.  The  incidence  of  AMS  using  the  paper  and 
pencil  and  electronic  versions  of  the  ESQ  at  RA  (0%  vs.  0%),  PP24  (35%  vs.  38%), 
PP48  (26%  vs.  26%),  and  PP72  (18%  vs.  18%)  were  also  not  significantly  different  in 
any  of  the  four  test  conditions.  The  relationships  between  the  severity  of  AMS 
calculated  from  the  two  versions  of  the  ESQ  at  RA  (r=0.43;p=0.01 ),  PP24 
(r=0.92:p=0.0001),  PP48  (r=0.82;p=0.0001),  and  PP72  (r=0.95;p=0.0001)  were  highly 
significant  (Figure  1 ).  The  relationships  between  the  incidence  of  AMS  calculated  from 
the  two  versions  of  the  ESQ  at  RA  (k=0.90:  p=0.01),  PP24  (k=0.90;  p=0.01),  PP48 
(k=0.91 ,  p=0.01 )  and  PP72  (k=0.92,  p=0.005)  were  also  highly  significant.  The  mean 
AMS-C  difference  scores  (paper  and  pencil  -  electronic)  in  each  of  the  four  test 
conditions  are  plotted  in  Figure  2.  The  mean  AMS-C  difference  scores  were  not 
significantly  different  from  zero  or  from  each  other  in  any  of  the  four  test  conditions. 
Bland-Altman  plots  for  the  individual  AMS-C  difference  scores  and  limits  of  agreement 
(±2SD)  in  each  of  the  four  test  conditions  are  presented  in  Figure  3.  The  level  of 
agreement  between  the  two  versions  of  the  ESQ  at  RA  and  PP72  was  exceptional  with 
95%  prediction  intervals  that  were  within  ±0.18  and  ±0.42  from  zero,  respectively.  The 
level  of  agreement  between  the  two  versions  of  the  ESQ  at  PP24  and  PP48  was 
moderate  with  95%  prediction  intervals  that  were  within  ±0.87  and  ±1 .09,  respectively 
from  zero. 
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Figure  1 .  Pearson-product  correlation  coefficients  for  the  relationship  between  AMS-C 
measured  from  the  paper  and  pencil  and  electronic  versions  of  the  Environmental 
Symptoms  Questionnaire  at  residence  altitude  (RA)  and  after  24-h  (PP24),  48-h 
(PP48),  and  72-h  (PP72)  exposure  to  4300  m  on  the  summit  of  Pikes  Peak  (PP) 
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Figure  3.  Bland-Altman  plots  for  the  mean  AMS-C  difference  scores  at  residence 
altitude  (RA)  and  following  24-h  (PP24),  48-h  (PP48),  and  72-h  (PP72)  exposure  to 

4300  m  on  the  summit  of  Pikes  Peak  (PP) 
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DISCUSSION 


The  purpose  of  this  study  was  to  validate  the  shortened  electronic  version  of  the 
ESQ  to  assess  AMS.  Our  major  findings  were  the  following:  1 )  neither  the  severity  nor 
the  incidence  of  AMS  differed  between  the  paper  and  pencil  and  electronic  versions  of 
the  ESQ  in  any  test  condition,  2)  the  relationships  between  both  the  severity  and 
incidence  of  AMS  measured  by  the  two  versions  of  the  ESQ  were  highly  significant,  and 
3)  the  AMS-C  difference  scores  between  the  two  versions  of  the  ESQ  in  all  four  test 
conditions  were  not  significantly  different  from  zero  or  from  each  other.  These  findings 
suggest  that  the  shortened  (11 -item)  electronic  version  can  be  substituted  for  the  67- 
item  paper  and  pencil  version  of  the  ESQ  in  order  to  assess  the  severity  and  incidence 
of  AMS  at  4300  m. 

Assessment  of  AMS  is  a  critical  component  in  most  studies  conducted  at  high 
altitude.  It  was  recognized  by  leaders  in  the  field  that  the  67-item  ESQ-III  was  lengthy 
and  cumbersome  to  use  in  high  altitude  research.  Thus,  a  consensus  group  proposed 
the  5-item  self-assessment  Lake  Louise  (LL)  scoring  system  in  1991 ,  which  was  revised 
in  1993  (23).  This  scoring  system  has  been  widely  used  since  that  time  but  our 
laboratory  (27)  and  others  (1 )  have  found  that  it  tends  to  overestimate  the  incidence  of 
AMS  compared  to  the  ESQ,  possibly  due  to  the  detection  of  AMS  at  an  earlier  stage  or 
in  a  milder  form.  As  such,  the  need  remained  for  a  shortened  and  less  cumbersome 
method  for  assessing  the  incidence  and  severity  of  AMS  that  directly  compares  to  the 
results  obtained  from  the  67-item  paper  and  pencil  ESQ.  Thus,  a  shortened  (1 1 -item), 
electronic  version  of  the  ESQ  was  developed  that  could  be  administered  on  a  hand¬ 
held  computer. 

Whenever  a  questionnaire  is  shortened  or  the  mode  of  delivery  is  altered,  it  is 
critical  to  validate  whether  these  changes  affect  the  values  of  the  measurement. 
Although  the  AMS-C  value  is  calculated  from  only  1 1  of  the  67  items  contained  in  the 
paper  and  pencil  ESQ  and  these  same  1 1  items  were  utilized  in  the  shortened 
electronic  version  of  the  ESQ,  it  was  unknown  whether  a  difference  in  the  number  of 
items  would  have  any  effect  on  AMS-C  values.  It  was  also  unknown  whether  changing 
the  mode  of  delivery  to  a  handheld  computer,  such  that  each  question  had  to  be 
answered  before  proceeding  to  the  next  question,  would  affect  AMS-C  values.  Given 
the  results  of  this  study,  it  does  not  appear  that  fewer  items  or  changing  the  mode  of 
delivery  affected  AMS-C  values  in  any  way.  The  fact  that  AMS-C  difference  scores 
were  not  significantly  different  in  any  test  condition  also  suggests  that  the  shortened 
electronic  version  of  the  ESQ  can  be  used  at  any  time  point  at  altitude. 

The  only  concern  with  our  analysis  of  AMS-C  values  measured  by  both  the 
paper  and  pencil  and  electronic  ESQ  was  the  wide  prediction  intervals  at  PP24  and 
PP48.  However,  close  examination  of  the  Bland  Altman  plots  at  PP24  and  PP48 
(Figure  3)  suggest  that  the  mean  AMS-C  points  along  the  x-axis  in  the  range  between 
0.5  and  1 .0  (the  threshold  level  for  AMS-C  is  >0.7)  are  not  widely  dispersed.  Thus,  the 
agreement  between  the  two  versions  of  the  ESQ  is  likely  much  higher  in  this  critical 
range  than  what  is  reported  for  the  overall  range  of  points.  It  is  only  when  the  AMS-C 
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values  get  larger  and  farther  away  from  typical  mean  values  of  AMS-C  that  the  points 
on  the  plot  become  more  widely  dispersed. 

Administering  the  questionnaire  by  computer  likely  insures  greater  accuracy 
because  the  possibility  of  leaving  questions  unanswered  or  making  stray  marks  does 
not  exist.  In  addition,  by  shortening  the  time  it  takes  to  answer  the  questionnaire, 
volunteer  motivation  is  more  likely  to  remain  high  such  that  volunteers  will  actually  take 
the  time  to  read  each  question  instead  of  just  randomly  circling  a  number.  In  other 
studies  that  have  compared  paper  and  pencil  administration  versus  computer 
administration  of  a  health  questionnaire,  no  differences  in  the  validity  of  answers  have 
been  reported  (4,  6,  14,  19,  20).  In  fact,  these  studies  reported  that  69%  to  82%  of  the 
users  preferred  computer  administration  of  the  test  (4,  6)  even  when  half  of  the  subjects 
reported  no  computer  experience.  In  addition,  one  study  reported  that  the  built-in 
requirement  to  answer  questions  in  the  computer  version  as  well  as  the  shortened 
administration  time  increased  both  accuracy  and  compliance  (19). 

CONCLUSIONS 

In  conclusion,  the  shortened  electronic  version  of  the  ESQ  represents  a  valid 
method  for  assessing  the  severity  and  incidence  of  AMS  during  72-h  of  exposure  to 
4300  m. 
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